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Abstract 

Practice sustainability is important to ensure that students have continued access to evidence- 
based practices. In this study, respondents from a national sample of 860 schools at varying 
stages of implementing school-wide positive behavioral interventions and supports (SWPBIS) 
were administered a research-validated measure of factors predicting sustained implementation 
of school-based practices. School demographic characteristics and specific school team actions 
were assessed to indicate which variables were most strongly associated with the four 
sustainability factors. Findings showed that in general, school demographic characteristics were 
not significantly related to sustainability. School team actions, especially the frequency of 
sharing data with the whole school staff, were statistically significantly related to sustainability. 
Implications for enhancing sustainability of school-based practices are discussed. 
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Variables Associated with Enhanced Sustainability of School-wide 
Positive Behavioral Interventions and Supports 

In a recent study of directions for school psychology research (McIntosh, Martinez, Ty, 

& McClain, 2013), prominent researchers indicated the growing need for empirical research in 
implementation science, the study of how practices are adopted, implemented, and sustained 
(Forman et al., 2013). Within implementation science, practice sustainability represents a 
significant challenge worthy of study. Practice sustainability can be defined as “a practice’s 
potential for durable implementation with high fidelity, when considering features of the 
practice, its implementation, and the context of implementation” (McIntosh & Turri, 2014, p. 
2062). Hence, sustainability references elements (including features of the practice itself, 
activities in the practice’s implementation, and the context of the implementation site) that make 
the practice more durable (Durlak & DuPre, 2008). As such, an important facet of measurement 
in implementation science is the difference between sustained implementation , which is the 
outcome, and sustainability, which is the presence of variables that predict sustained 
implementation. The literature base indicates a range of variables linked to sustained 
implementation (Domitrovich et al., 2008; Fixsen, Blase, Duda, Naoom, & Van Dyke, 2010; 
Gersten, Chard, & Baker, 2000; Han & Weiss, 2005; McIntosh, Filter, Bennett, Ryan, & Sugai, 
2010). However, it is important to identify which of these variables are most strongly related to 
the potential for durable implementation at strong fidelity. Identifying these pivotal variables can 
help researchers and practitioners predict and prevent practice abandonment, which is costly in 
terms of resources, morale, and most importantly, student outcomes (Hume & McIntosh, 2013). 
School Demographic Characteristics as Potential Predictors of Sustained Implementation 



VARIABLES ASSOCIATED WITH SWPBIS SUSTAINABILITY 


4 


One set of variables that have been noted as potentially important to sustainability (and 
therefore worthy of further study) are school demographic characteristics, particularly those 
related to racial or socioeconomic diversity, or differences in school structure, such as school 
size or grade levels served (Forman et al., 2013). School demographic characteristics are 
hypothesized as important because some characteristics (e.g., low community socioeconomic 
status [SES]) are associated with greater risk of inconsistent implementation or abandonment 
(Bradshaw & Pas, 2011), and practices developed or tested with primarily white, middle class 
student samples may not be as effective for the diverse student populations seen in many U.S. 
schools (Durlak & DuPre, 2008; Forman et al., 2013; D. C. Gottfredson, 2001). In addition, high 
schools are often regarded as more challenging contexts for implementing school-wide 
interventions, based on their structure, staff size, and strong content focus (Flannery, Sugai, & 
Anderson, 2009). 

The limited empirical research on school characteristics as predictors of sustained 
implementation is inconsistent. One study found that both low SES and high minority student 
body composition were factors related to low implementation and sustainability of a school- 
based cognitive-behavioral intervention (G. D. Gottfredson, Jones, & Gore, 2002). A national 
study of school violence prevention interventions found that high enrollment (i.e., school size), 
urbanicity (i.e., community population density), and low SES were significant predictors of low 
levels of implementation but were no longer significant when considering program 
characteristics, such as standardization of materials and integration into typical operating 
practices (Payne & Eckert, 2010). 

In addition to research regarding school-based interventions in general, a number of 
recent studies have specifically examined predictors of implementation of school-wide positive 
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behavioral interventions and supports (SWPBIS; Sugai, Horner, & McIntosh, 2008). SWPBIS is 
a widely used approach for promoting prosocial behavior and reducing problem behavior. 
Bradshaw and Pas (2011) found that high student enrollment (but not low SES) was a significant 
barrier to SWPBIS implementation. Molloy and colleagues (2013) found that grade levels served 
(i.e., high schools), high enrollment, and low SES were negatively related to implementation. 
Frank and colleagues (2009) found that SES was unrelated to implementation, but 
implementation was higher for schools with moderate racial diversity (25 to 50% non-white) 
than for schools with high or low diversity. Given these inconsistent findings to date, school 
demographic characteristics deserve further study as potential predictors of sustained 
implementation. 

School Team Actions as Potential Predictors of Sustained Implementation 

Even considering the small research base on school demographic characteristics, there is 
even less research on school implementer actions as they relate to SWPBIS implementation. 
Existing research has focused more on provider characteristics (e.g., levels of training, self- 
efficacy of school personnel) or the importance of a team-based approach in general than specific 
strategies that school teams can take after adoption (Durlak & DuPre, 2008; Payne & Eckert, 
2010). Research investigating school team access to coaching as predictive of implementation is 
inconclusive (Bradshaw & Pas, 2011; Domitrovich et al., 2008). Some research has indicated 
that simply whether the school team uses data for decision making is a particularly important 
predictor of sustainability of SWPBIS (Coffey & Homer, 2012; McIntosh, Mercer, et al., 2013), 
but effects of the frequency of specific team actions (as opposed to whether they are used at all) 
has not been assessed to date. It is possible that the frequency of these actions—rather than their 
presence—are stronger predictors of sustained implementation. 
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Purpose of the Study 

The purpose of this study was to assess the extent to which school demographic 
characteristics and frequencies of school team actions were associated with increased likelihood 
of sustained implementation of SWPBIS. Such research has the potential to inform researchers 
and practitioners regarding the most important variables to target to enhance implementation and 
sustainability of school-based interventions. The current study used a large sample, a research- 
validated measure of contextual elements predicting sustained implementation, and a range of 
predictors to extend previous sustainability research into what structural and functional 
facilitators are most related to sustainability. The use of multiple predictors within a structural 
equation modeling approach allowed assessment of each variable’s unique association with 
potential for sustained implementation. 

Method 

Participants and Settings 

The study assessed a total of 860 schools across 14 states implementing SWPBIS. One 
individual with knowledge regarding each school’s SWPBIS systems (e.g., School SWPBIS 
team member, district coach) participated for each school. Table 1 presents the most recent 
National Center for Education Statistics demographic data available for schools (99%, n = 851). 

Measures 

Practice sustainability. The School-wide Universal Behavior Sustainability Index: 
School Teams (SUBSIST; McIntosh, Doolittle, Vincent, Homer, & Ervin, 2009) is a measure of 
factors predicting sustained implementation of school-wide behavior support interventions that 
has been validated for this purpose. Respondents rate the extent to which each variable is present 
at that time in their school for 39 items, with responses in a 4-point Likert-type scale (e.g., 1 = 
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not true , 4 = very true). The SUBSIST’s psychometric properties have been studied extensively, 
including a content validity study with an expert panel (content validity index = .95), a pilot 
study (a = .87, interrater reliability = .95, two-week test-retest reliability = .96), factor analytic 
studies with two separate samples, and studies with three separate samples showing strong 
prediction of sustained SWPBIS implementation (r range = .45 to .68; McIntosh et al., 2011; 
McIntosh, Mercer, et al., 2013; Mercer, McIntosh, Strickland-Cohen, & Horner, 2014; Runge, 
Gorlaski, & Wagner, 2013). Factor analyses have shown a four-factor structure, with two school- 
level factors, School Priority (20 items, a = .94) and Team Use of Data (11 items, a = .94) and 
two district-level factors, District Priority (5 items, a = .71) and Capacity Building (3 items, a = 
.74). 

School demographic characteristics. Data on school demographic characteristics were 
used to assess associations with SUBSIST factors. Self-reported years implementing SWPBIS 
was divided into groups of 0 to 1 years, 2 to 4 years, and 5 or more years to indicate stages of 
implementation identified by Adelman and Taylor (initial implementation, institutionalization, 
and ongoing evolution; 1997). Data from the National Center for Education Statistics included 
grade levels served (elementary, middle, or high school), enrollment, urbanicity (on a scale of 1 
to 4, using the federal categories of rural, town, suburb, and city), percent of non-white students, 
and percent of students receiving free and/or reduced price lunch. 

School team actions. As part of the SUBSIST administration, respondents were asked to 
self-report the frequency of three sets of actions. These actions were (a) frequency of SWPBIS 
team meetings (on a 5-point scale from every other month to weekly), (b) frequency of sharing 
data with the whole school staff (on a 10-point scale from less than once per year to weekly), and 
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(c) hours of SWPBIS coaching received (on a 7-point scale from none to more than 5 hours per 
week). 

Procedure 

Data for the current study came from the first year of a 3-year project examining 
implementation and sustainability of SWPBIS. For recruitment, state SWPBIS teams shared 
details at training events or sent emails to schools either implementing or preparing to implement 
SWPBIS through state or district training events. A member of each school team participated by 
completing the SUBSIST online. Demographic data were obtained through the NCES website. 
Data Analyses 

Structural equation modeling in M plus 7.1 (Muthen & Muthen, 2012) was used to assess 
each variable’s unique association with the four latent factors of the SUBSIST, using the mean- 
and variance-corrected weighted least squares (WLSMV) estimator. Standard errors were 
corrected for nesting of schools within districts using the M plus COMPLEX command 
(Asparouhov, 2005), and continuous predictors were centered at the grand mean. Model fit was 
assessed through examining the Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), and 
Root Mean Square Error of Approximation (RMSEA), using the common criteria of CFI > .95, 
TLI > .95, and RMSEA < .06 (Hu & Bentler, 1999). Missing data (6% of the data, including 
“don’t know/NA”) were handled using multiple imputation with 1000 M plus datasets. Given the 
large sample size, and 4 DVs, the alpha level was set conservatively at a = .001. 

Results 

Intercorrelations among predictors are provided in Table 2. Fit for the structural equation 
model was adequate on all indicators other than chi-square, x~ (1011) = 2098.026, p < .001, CFI 
= .923, TLI = .914, RMSEA = .035. Table 3 presents the unstandardized beta weights for the 
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regressions for each latent SUBSIST factor. For School Priority, significant predictors included 
years implementing SWPBIS, grade levels served, and frequency of data sharing with staff. For 
Team Use of Data, significant predictors were years implementing SWPBIS, grade levels served, 
frequency of team meetings, and frequency of data sharing with staff. For District Priority, the 
significant predictor was frequency of data sharing with staff. For District Capacity Building, 
significant predictors included frequency of data sharing with staff and access to coaching. 

Discussion 

In this study, school demographic characteristics and school team actions were assessed 
for their relation with a research-validated measure of sustainability for 860 schools 
implementing SWPBIS. Although grade levels served and years implementing were both 
significant predictors of school-level factors, other school demographics, such as percent of 
students receiving free or reduced lunch and percent of non-white students, were not 
significantly related to sustainability. Frequency of team meetings and access to coaching were 
each related to one sustainability factor, whereas the frequency of teams sharing data with all 
school staff was significantly related to all four sustainability factors. 

Results regarding school demographics were consistent with existing SWPBIS research 
indicating little to no effects of these variables on implementation. It is possible that challenging 
contexts (e.g., low SES) may provide the motivation to adopt and sustain effective practices 
because they are seen as instrumental in achieving valued outcomes and avoiding experiences of 
failure (Andreou, McIntosh, Ross, & Kahn, in press; Baker, Gersten, Dimino, & Griffiths, 2004). 
In support of this hypothesis, Bradshaw and Pas (2011) found that schools experiencing higher 
rates of suspension, mobility, and academic failure were significantly more likely to adopt 
SWPBIS. In addition, the non-standardized framework of SWPBIS allows adaptation for 
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culturally and linguistically diverse student bodies, with substantial decreases in problem 
behavior in schools with large non-white populations (Bohanon et al., 2006; McIntosh, Moniz, 
Craft, Golby, & Steinwand-Deschambeault, 2014; Scott, 2001) Additional support for this point 
comes from research showing statistically significantly lower discipline disproportionality for 
schools implementing SWPBIS (Vincent, Swain-Bradway, Tobin, & May, 2011). 

The findings that high schools in particular had lower scores on school-level 
sustainability factors is consistent with descriptive literature regarding the challenges associated 
with implementing SWPBIS in high schools, such as cultivating and maintaining staff support 
(Flannery et al., 2009). However, recent research indicates that implementation to criterion is 
possible and leads to significant reductions in problem behavior compared to control schools 
(Flannery, Fenning, Kato, & McIntosh, 2014). Interestingly, high schools had much higher 
district-level sustainability factor scores. Because high schools are often the last schools in a 
district that implement SWPBIS, we hypothesize that this delay in implementing may benefit 
high schools because there are already solid district SWPBIS systems in place, leading to district 
policies, resources, and training and coaching structures that support school level 
implementation. As a result, high school SWPBIS teams may be able to rely on more solid 
district systems to overcome challenges in their implementation. 

In contrast to school demographic characteristics, school team actions were more strongly 
associated with sustainability, especially the frequency of sharing of data with the entire school 
staff, the only significant predictor of all four sustainability factors. In addition to continuing to 
document the importance of the use of data for sustainability (Coffey & Homer, 2012; McIntosh, 
Mercer, et al., 2013), these results point to the importance of a specific action for teams to take to 
enhance sustainability. In addition to the team’s general use of data (which is one of the 
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sustainability factors), the actual frequency of sharing the data and the decisions based on them 
with the entire staff on a regular basis may enhance sustainability. 

Although the SUBSIST’s Team Use of Data factor includes an item assessing whether 
data are presented to all school personnel at least quarterly, the strong association between the 
actual frequency of sharing data with staff and all four sustainability factors is important to note. 
As seen in Figure 1, this frequency was strongly related to overall sustainability. This finding 
points to a potential mechanism in sustainability. By sharing data frequently with school staff, 
the school team may enhance not just data-based decision making, but also the priority of the 
practice for staff and administrators and strengthen perceptions that implementation leads to 
valued outcomes. However, it is important to note that although sharing data with staff may seem 
like a pivotal strategy in sustainability, more research is needed, due to the correlational nature of 
this study. It is possible that sharing data with staff is more accurately regarded as an outcome of 
administrator support, collection of data, and skills in interpreting data than an effective 
intervention in of itself. As a result, further research is needed to assess whether data sharing 
itself is causally related to sustainability. 

[Insert Figure 1 about here] 

Access to coaching was a significant predictor for District Capacity Building, which is 
logical because coaching is a primary method of supporting schools in implementation, but it 
was not a predictor of the school-level factors. This finding is consistent with some previous 
SWPBIS research, in which in general, the amount of coaching is related to sustained 
implementation, but not when other factors are considered (Bradshaw & Pas, 2011; Mathews, 
McIntosh, Frank, & May, 2014). One interpretation is that coaching is only a strong predictor 
when it is effective (e.g., related to improved implementation quality by the team), but not when 
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it does not improve practices. For example, schools that are struggling may receive additional 
coaching, which may falsely indicate that coaching is negatively related to outcomes. 

Limitations and Future Research 

These results should be interpreted in light of a few key limitations that should be 
validated through future research. Although three previous studies have shown the SUBSIST 
factors to be strongly related to actual fidelity of implementation of SWPBIS, the outcome was 
the potential for sustained implementation, not sustained implementation itself. These results 
should be viewed as tentative until they are further replicated with longitudinal fidelity of 
implementation. In addition, relying on schools that elect to participate in a national study may 
have biased the responses. Finally, these results are most applicable to sustainability of SWPBIS, 
and they may not generalize to other school practices. Future research should assess the relations 
among school demographics, school team actions, and sustainability for a range of school-based 
interventions, such as academic RTI initiatives, social-emotional learning programs, and school 
mental health practices. It may be possible that different school characteristics or team actions 
are more predictive of sustainability for these practices. 
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Table 1 


School Demographic Data 


Demographic Variable 

n 

(N =860) 

Mean or % (SD) 

Enrollment 

851 

537.18 (341.801) 

% of Students Receiving FRL 

847 

49.7% (23.4%) 

% of Non-white Students 

849 

39.9% (30.5%) 

Grade Level 

825 


Primary 

562 

68.1% 

Middle 

165 

20.0% 

High 

98 

11.9% 

Urbanicity 

851 


Rural 

211 

24.8% 

Town 

122 

14.3% 

Suburb 

279 

32.8% 

City 

239 

28.1% 

Years Implementing SWPBIS 

860 


0 to 1 Years 

212 

24.7% 

2 to 4 Years 

410 

47.7% 

5 or more 

238 

27.7% 


Note. School demographic data obtained from National Center for Education Statistics for 99% 


of schools. * FRL = free and/or reduced lunches. 
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Table 2 


Intercorrelations among Predictor Variables 



1 

2 

3 

4 

5 

6 

7 

8 

9 

1. Years 

Implementing 

- 

-.053 

.050 

.081* 

.050 

-.070* 

_ 102** 

-.003 

.041 

2. Grade Levels 


- 

.371** 

.109** 

-.048 

-.106** 

.068 

.029 

.004 

3. Enrollment 



- 

.183** 

.196** 

-.172** 

-.059 

-.084* 

.082 

4. Urbanicity 




- 

.550** 

.219** 

-.077* 

- U3** 

-.027 

5. % Non-white 





- 

.512** 

_ 143** 

. 148** 

-.025 

6. % Free/Reduced 






- 

-.001 

-.029 

-.077 

7. Team Meetings 







- 

.154** 

-.035 

8. Data Sharing 








- 

.052 

9. Coaching 









- 

Note, n range = 776-860. 

>< .05. * 

><.01. 

"> < .001. 
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Table 3 

Results of Latent Regression Analyses Predicting Each Sustainability Factor 


Sustainability Factor 


School Team Use of District Priority District Capacity 

Priority Data Building 


Predictor 

B 

B 

B 

B 

Years Implementing 

0.103* 

0.166* 

0.018 

0.028 

Grade Levels 

-0.150* 

-0.157* 

-0.073 

- 0.111 

Enrollment 

-0.066 

-0.028 

-0.002 

-0.106 

Urbanicity 

-0.034 

-0.062 

-0.095 

-0.108 

% of Non-white 

-0.036 

-0.126 

0.279 

0.279 

Students 





% of Students with 

-0.017 

-0.031 

-0.132 

-0.411 

Free/Reduced Meals 





Frequency of Team 

0.038 

0.114* 

0.021 

0.112 

Meetings 





Frequency of Sharing 

0.073* 

0.162* 

0.064* 

0.079* 

Data with Staff 





Access to Coaching 

0.001 

0.006 

0.045 

0.094* 

Variance Explained 

Rf 

Rf_ 

Rf 

Rf 


.223* 

.367* 

.137* 

.211* 


Note, n = 860. p < .001. Unstandardized beta weights presented. 
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Figure 1. Associations between Frequency of Data Sharing and Overall Sustainability 



